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ABSTRACT 

\ 

The  performance  of  a computer-aided  detection  model  has 
been  validated  using  a large  set  of  recorded  sea  test  data.  The 
data  base,  processing  procedures  and  results  obtained  are  described. 
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model  is  described  in  Appendix  A.  Appendix  B provides  a 
derivation  of  the  transformation  from  peak  height  to  log  likelihood 
ratio.  A simplified  model  for  predicting  CAD  performance  is 
described  in  Appendix  C.  Appendix  D provides  several  sets  of 
displays  obtained  with  sea  data  to  give  a visual  presentation  of 
the  effect  of  the  CAD  model  on  a display. 

The  results  obtained  in  this  study  indicate  that  the 
CAD  model  can  detect  submarine  target  tracks  with  a signal-to- 
noise  ratio  at  the  output  of  the  signal  processor  as  low  as  11  dB. 
This  performance  compares  favorably  with  that  obtained  by  a sonar 
operator.  The  data  rate  required  to  display  the  output  of  the  CAD 
model  is  substantially  reduced  relative  to  that  required  to 
directly  display  the  output  of  the  signal  processor. 
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2.  DESCRIPTION  OF  DATA  PROCESSING  PROCEDURES 


(U)  The  data  processing  procedures  Implemented  to  accomplish 
the  validation  of  the  computer-aided  detection  (CAD)  model  are 
described  in  the  following  sections.  This  chapter  concentrates 
on  the  procedures  applied  external  to  the  basic  CAD  model.  The 
internal  data  processing  procedures  of  the  CAD  model  are 
described  in  detail  in  Appendix  A. 

(U)  To  provide  a starting  poiht  for  the  description  of  the 
data  processing  procedures,  the  sea  test  data  base  used  is 
described  in  Section  2.1.  Section  2.2  describes  the  functional 
data  flow  within  the  processing  procedures,  and  is  organized  to 
parallel  the  order  in  which  basic  computer  runs  were  executed 
to  accomplish  the  processing  procedures. 

2.1  DATA  BASE  DESCRIPTION 

(C)  Data  gathered  during  the  technical  evaluation  of  the 
AN/SQS-26  sonar  was  used  as  the  data  base  for  the  validation  of 
the  CAD  model.  The  TECHEVAL  data  included  about  4,000  ping 
cycles  divided  into  about  200  runs  of  about  20  ping  cycles  per 
run.  From  this  data  base  about  1250  ping  cycles  were  selected 
and  processed  through  the  CAD  model.  The  output  of  the  shipboard 
signal  processing  unit  (and  other  information)  was  recorded  on 
1 inch  14  channel  instrumentation  tape  by  an  analog  data 
acquisition  system  (ADAS) . The  recorded  analog  information  was 
converted  to  digital  format  as  part  of  the  data  processing 
accomplished  in  support  of  TECHEVAL.  The  digitized  data  was 
formatted  to  have  one  digital  file  for  each  ping  cycle.  Within 
each  digital  file,  the  data  consisted  of  samples  taken  during 
two  time  gates.  The  first  time  gate  occurred  during  transmit 
and  was  approximately  *5  second  long.  The  second  time  gate 
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occurred  during  receive,  and  was  approximately  12  seconds  long. 
The  position  of  the  second  time  gate  was  controlled  by  the 
sonar  operator's  cursor.  In  this  way  the  12-second  interval 
always  overlapped  the  target  signal  and  the  transponder  signal. 

(U)  During  each  time  gate  data  was  sampled  at  a 1 kHz 
rate  from  nine  analog  channels.  A time  code  channel,  recorded 
on  the  analog  tape,  also  was  decoded  and  stored  in  the 
digital  file  at  a 1 kHz  rate.  The  nine  analog  channels  which 
were  sampled  were: 


(1) 

SSI 

(2) 

CW  AGC 

IN 

(3) 

CW  IN 

(4) 

CW  OUT 

(5) 

CP  AGC 

IN 

(6) 

CP  IN 

(7) 

CP  OUT 

(8) 

3-BIT  CP  OUT 

(9) 

DELTIC 

REF. 

(U)  For  the  program  to  validate  the  CAD  model,  the  only 
information  that  was  needed  from  the  digital  files  was; 

(U)  (1)  Time  Codes.  By  comparing  the  time  codes  in  the 

first  gate  with  those  in  the  second  time  gate  the  time  of  signal 
arrival  relative  to  the  time  of  transmit  was  determined. 


(U)  (2)  CP  OUT.  The  output  of  the  CODED  PULSE  processor 

was  used  as  the  basic  input  to  the  CAD  model. 


(C)  In  addition  to  the  sonar  data  discussed  above,  the 
sonar  operator  recorded  a response  to  the  signal  received  on 
each  ping.  The  responses  were;  NO  SIGNAL,  WEAK,  MEDIUM,  aind 
STRONG.  These  operator  responses  were  punched  on  cards  for 
use  in  the  single  ping  comparison  of  CAD  performance  with 
operator  responses. 
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2.2  FUNCTIONAL  DATA  FLOW 

(U)  The  functional  data  flow  is  best  discussed  in  terms 
of  four  smaller  subsections.  Sections  2.2.1  and  2.2.2  are 
concerned  with  the  data  reduction  necessary  to  produce  the 
absolute  measure  of  CAD  model  performance  described  in  Section 
2.2.3.  Comparison  of  the  CAD  model  performance  with  operator  t 

performance  is  discussed  in  Section  2.2.4. 

2.2.1  Phase  I Data  Reduction 

(U)  Before  the  TECHEVAL  data  described  in  Section  2.1 
could  be  used  in  the  CAD  validation  process  some  preliminary 
data  reduction  was  necessary.  Figure  1 shows  a condensed 
description  of  the  data  reduction  process  described  below. 

(U)  The  calibration  data  processing  calculates  the  mean 
and  standard  deviation  of  the  noise  present  using  data  from 
the  first  two  echo  cycles  of  the  run.  The  calculated  values  of 
mean  and  standard  deviation  are  passed  to  the  thresholding 
process . 

(U)  The  thresholding  process  uses  a log  likelihood  ratio 
(see  Appendix  A)  threshold  to  produce  an  array  containing  the 
value  (in  log  likelihood  ratio)  and  location  of  each  peak  that 
exceeds  this  threshold.  This  processing  condenses  the  large 
volume  of  uniform  time  function  samples  by  extracting  peak 
sample  events,  and  uses  the  mean  and  standard  deviation  obtained 
from  the  calibration  data  to  transform  these  peak  amplitudes  to 
a normalized  quantity,  in  this  case  log  likelihood  ratio. 

(U)  It  is  evident  that  this  thresholding  process  is  more 
involved  than  the  normal  thresholding  process.  The  data  is  first 
scanned  to  select  the  local  peaks.  (A  sample  value  Xj^  is  a local 
peak  if  and  only  if  x^  > and  Xj^  & ^i+1^  * When  two  local 
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peaks  occur  within  a single  resolution  interval  of  the  sonar, 
the  ambiguity  is  resolved  by  selecting  the  larger  of  the  two. 
Each  local  peak  is  then  converted  to  a log  likelihood  y^ 
through  the  relation 

x,-u 

Yi  = 5-)  + b . 

where 

U = the  mean  value  of  the  data  calculated  from 
the  calibration  data. 

a = the  standard  deviation  of  the  data  calculated 
from  the  calibration  data, 

a and  b = the  conversion  constants  for  the  log  likelihood 
ratio  L,  L = ax  + b. 

The  numerical  values  used  for  a and  b are 
2.45  and  -5.2  respectively.  These  values 
are  derived  in  Appendix  B. 

(U)  After  the  log  likelihood  ratio  conversion  the 
thresholding  process  uses  the  time  code  data  to  express  the 
position  of  each  local  peak  as  an  arrival  time  relative  to  the 
transmit  time  of  the  signal.  Thus,  the  entries  in  the  array 
produced  by  this  process  consist  of  normalized  peak  amplitudes 
expressed  in  units  of  log  likelihood  ratios  and  corresponding 
peak  positions  expressed  in  milliseconds  relative  to  the 
transmit  time. 

(U)  The  collection  process  collects  the  peak  amplitudes 
and  positions,  generated  ping  by  ping  in  the  thresholding 
process,  into  one  large  array  containing  information  about  the 
entire  run.  The  resulting  array  is  stored  onto  magnetic  tape 
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for  later  processing,  and  is  also  processed  to  obtain  a 
computer  generated  display.  Figure  2 is  an  example  of  this 
type  of  display.  The  numbers  down  the  left  side  of  the  display 
identify  the  ping  cycles.  The  horizontal  displacement 
represents  time  of  arrival  or  range.  The  digits  within  the 
display  indicate  dB  in  excess  of  some  given  signal-to-noise 
ratio  value  shown  in  the  display  heading.  The  purpose  of 
this  display  is  to  verify  visually  the  track  chosen  by  the 
automatic  track  localization  process  described  below. 

(U)  To  this  point  the  information  stored  in  the  array 
produced  by  the  collection  process  includes  both  signal  peaks 
and  noise  peaks.  In  order  to  produce  the  desired  measure  of 
CAD  performance,  it  is  necessary  to  separate  the  signal 
information  from  the  noise  information.  The  automatic  track 
localization  process  is  implemented  to  obtain  this  separation. 

(U)  In  the  automatic  track  localization  process,  peaks 
belonging  to  a signal  track  are  extracted  from  the  input  array 
which  is  provided  by  the  collection  process.  Since  the  noise 
and  signal  peaks  input  to  the  process  were  obtained  from  N 
consecutive  pings,  a maximum  of  N signal  peaks  and  positions 
may  define  a single  track.  When  the  track  does  not  contain  a 
signal  peak  on  one  or  more  pings,  there  will  be  fewer  than  N 
peaks.  Many  pings,  however,  will  contain  more  than  one  signal 
peak  that  oould  be  associated  with  a single  track.  The 
automatic  track  localization  process  uses  a multiple-step 
procedure  to  select  a single  peak  from  each  ping  for  a given 
track. 


(U)  The  first  step  consists  of  locating  the  largest  peak 
in  the  entire  input  array.  An  initial  assumption  is  made  that 
this  largest  peak  ( called  the  "pivot"  peak)  belongs  to  the 
track.  All  peak  positions  and  amplitudes  on  each  ping 
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(excluding  the  ping  containing  the  pivot  peak)  are  then  examined 
and  an  cunplitude-weighted  slope  density  function  is  calculated. 
The  slope  from  each  peak  position  to  the  pivot  peak  position 
contributes  to  the  density  function  by  an  amount  related  to 
the  amplitude  of  the  peak.  After  all  peaks  have  been  examined, 
the  slope  corresponding  to  the  maiximum  of  the  density  function 
is  selected  and  is  used  to  define  a straight  line  through  the 
pivot  peak.  This  straight  line  is  the  first  approximation  to 
the  track. 

(U)  The  straight  line  track  approximation  is  used  to 
obtain  an  initial  set  of  signal  peaks,  which  may  initially 
include  more  than  N peaks.  A signal  peak  on  a given  ping  is  a 
candidate  for  inclusion  in  the  track  if  its  amplitude,  degraded 
by  the  deviation  of  its  position  from  the  straight  line 
approximation,  exceeds  a set  threshold.  To  reduce  the  number 
of  peaks  in  the  set  to  N (or  less) , the  largest  such  peak  on 
each  ping  is  selected  to  form  the  initial  set  of  signal  peaks. 

(U)  The  remainder  of  the  automatic  track  localization 
process  consists  of  an  iterative  procedure  in  which  successive 
least  squares  quadratic  fits  are  made  to  signal  peak  positions 
beginning  with  the  initial  set  of  signal  peaks  described  above. 
After  each  quadratic  fit  has  been  determined,  the  input  array 
of  peaks  is  examined  to  see  which  peaks  belong  to  the  track 
defined  by  the  quadratic  curve.  At  this  point  one  or  more  of 
the  initially-selected  peaks  may  be  replaced  by  other  peaks 
which  are  closer  to  the  curve.  The  peaks  selected  then  become 
the  basis  for  the  next  curve-fitting  step.  The  process 
terminates  when  two  successive  quadratic  fits  define  tracks 
containing  the  same  set  of  signal  peaks. 
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(U)  At  the  termination  of  the  automatic  track  localization 
process,  the  signal  peak  amplitudes  and  positions  defining  the 
track  are  produced  in  an  output  array.  The  data  is  stored  onto 
a magnetic  tape  to  be  processed  later  and  is  processed  to  obtain 
a computer  generated  display  to  illustrate  the  selected  track. 
From  Fig.  3 it  is  seen  that  the  format  of  this  display  is 
exactly  the  same  as  that  of  the  previously  described  display. 
Therefore,  comparison  of  the  two  displays  to  verify  the  results 
of  the  automatic  track  localization  process  is  possible. 

2.2.2  Phase  II  Data  Reduction 

(U)  The  next  step  in  the  validation  process  is  to  put  the 
data  into  a form  such  that  an  absolute  measure  of  CAD 
performance  can  be  obtained. 

(U)  To  produce  the  absolute  performance  measure  two  types 
of  curves  are  needed: 

(U)  (1)  False  alarm  rate  curves  as  a function  of  the 

likelihood  ratio  threshold,  and 

(U)  (2)  Required  signal-to-noise  ratio  for  50%  probability 

of  detection  as  a function  of  the  likelihood  ratio  threshold. 


The  false  alarm  rate  curves  can  be  generated  by  processing  a 
large  volume  of  sea  data  noise  through  the  C7UD  model.  Required 
signal-to-noise  ratio  curves  are  generated  from  the  signal 
data  that  has  been  processed  by  the  CAD  model.  From  the  above 
brief  discussion  of  the  two  types  of  information  needed  it  is 
evident  that  it  will  be  necessary  to  remove  the  signal  from  the 
signal  plus  noise  data  before  any  further  processing  can  be 
accomplished. 

(U)  To  aid  in  the  discussion  of  the  Phase  II  data  reduction 
a block  diagram  is  provided  in  Fig.  4. 
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FIG.  4 BLOCK  DIAGRAM  OF  PHASE  II  DATA  REDUCTION 


13 


UNCLASSIFIED 


CONFIDENTIAL 


14 


(U)  The  condensed  slcrnal  and  noise  data  consists  of  the 
amplitudes  and  location  values  produced  by  the  collection  process 
discussed  earlier.  From  the  automatic  track  localization 
process,  also  described  in  the  previous  ^section,  the  cimplitudes  and 
locations  of  the  saumples  contributing  to  a track  are  known.  The 
track  deletion  process  uses  this  condensed  signal  data  to  remove 
the  signal  track  from  the  signal  plus  noise  array. 

(U)  Because  of  the  sampling  techniques  used  in  digitizing 
the  TECHEVAL  data,  the  submarine  track  usually  occurs  about  1.5 
seconds  from  the  beginning  of  the  cursor  gate.  Multiple  paths 
to  the  target  often  exist,  causing  multiple  parallel  tracks. 

Because  these  multiple  tracks  are  usually  confined  to  the  first 
five  seconds  of  the  cursor  gate,  these  first  five  seconds  are 
not  used  in  developing  false  alarm  rate  curves.  This  procedure 
successfully  eliminates  most  target  tracks.  As  a further 
precaution,  in  processing  the  data  for  false  alarm  rate  curves, 
any  track  which  integrates  to  a log  likelihood  ratio  of  fourteen 
is  called  as  a target  track  and  is  not  counted  in  the  noise 
false  alarm  statistics. 

(U)  The  resulting  noise  only  data  is  catalogued  by  the 
sorting  process  to  obtain  a tabular  representation  of  the  noise 
probability  density  function.  This  table  is  output  to  magnetic 
tape  for  later  processing.  The  noise  only  data  is  also  processed 
through  the  CAD  model  and  into  the  sorting  process  to  obtain 
noise  probability  density  function  applicable  to  the  output  of  the 
CAD  model.  Likewise,  this  table  is  output  to  magnetic  tape 
for  later  processing. 

(C)  As  explained  above,  the  probability  of  detection 
curves  are  produced  from  the  results  obtained  by  processing 
signals  through  the  CAD  model.  Details  concerning  the  CAD  model 
may  be  found  in  Appendix  A.  For  discussion  here,  it  is 
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sufficient  to  state  that  the  CAD  model  implements  ping-to-ping 
integration  of  the  logarithm  of  the  likelihood  ratio  along 
consistent  target  tracks.  The  expected  track  strength  (or 
track  likelihood  ratio)  increases  as  the  number  of  pings 
increases.  This  result  is  consistent  with  results  obtained 
with  optical  ping-to-ping  integration. 

(C)  From  the  discussion  above,  it  is  apparent  that 
the  probability  of  detection  associated  with  a ping-to-ping 
integrating  system  depends  on  the  number  of  pings  for  which 
integration  is  accomplished  as  well  as  on  the  average  signal- 
to-noise  ratio.  It  is  also  evident  that  the  performance  of 
the  CAD  model  varies  with  respect  to  the  number  of  pings  the 
model  is  allowed  to  access.  For  this  reason,  to  adequately 
evaluate  the  CAD  model  it  was  necessary  to  vary  the  number 
of  pings  to  be  integrated,  and  thus  determine  the  model's 
performance  when  allowed  to  access  3 pings,  4 pings,  and  so 
forth. 

(U)  As  mentioned  earlier,  the  TECHEVAL  data  consisted 
of  approximately  20  pings  per  run.  The  CAD  model  was  modified 
to  enable  it  to  clear  its  memory  after  each  sequence  of  N 
pings  was  processed  in  order  to  simulate  having  had  access  to 
only  those  N pings. 

(U)  One  way  to  process  these  sequences  would  have  been 
to  determine  the  CAD  performance  individually  for  each  value 
of  N.  However,  in  the  interest  of  efficiency,  all  the  sequences 
were  processed  through  the  CAD  model  each  time  a run  was 
processed.  The  eight  CAD  model  output  channels  shown  in  Fig.  4 
contain  information  obtained  by  allowing  the  model  to  have 
access  to  different  numbers  of  pings  as  shown  on  the  output 
labels,  1,  2,  3,  4,  5,  6,  8,  and  12. 
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(U)  The  CAD  performance  monitor  uses  CAD  model  input  and 
output  data  to  obtain  information  packages  consisting  of  four 
entries : 

(U)  (1)  N,  the  number  of  pings  for  which  integration  was 

performed. 

(U)  (2)  LLR,  the  maiximum  track  log  likelihood  ratio 

obtained  for  the  N pings  processed. 

(U)  (3)  S/il,  the  estimated  value  of  signal-to-noise 

ratio  for  the  N-ping  sequence  going  into  the  CAD  model. 

(U)  (4)  S/N;MAX,  the  maximum  signal-to-noise  ratio 

which  occurred  in  the  N-ping  sequence  going  into  the  CAD  model. 

(U)  The  estimated  signal-to-noise  ratio  refers  to  a 
signal-to-noise  ratio  of  the  form 


P-x 

a 


where 

P is  the  signal  peak, 

X is  the  noise  mean, 

a is  the  standard  deviation  of  the  noise. 

If  the  signal  was  detected  on  all  N pings  (i.e.  no  missed 
pings) , then  the  estimated  signal-to-noise  ratio  was  a 
straightforward  average.  If  some  pings  occurred  which  did  not 
contain  signals  the  average  signal-to-noise  ratio  was  calculated 
as , 

K 

5 = 5 t ^ '5>i  " O'-")  '5>est’  ■ 

i=l 
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where 


the  number  of  pings  which  contained  a 
detected  signal, 

the  signal-to-noise  ratio  for  the  pings  which 
contained  a detected  signal, 

an  estimate  of  the  signal-to-noise  ratio 

for  the  missed  pings.  The  value  of  (^) 

is  calculated  by  a table  lookup  using 
N-K 

N • 
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3.  DESCRIPTION  OF  RESULTS  OBTAINED  AFTER 
PROCESSING  SEA  DATA 


(U)  This  chapter  is  concerned  with  a discussion  of  the 
results  obtained  using  the  processing  procedvil'es  described  in 
the  previous  chapter  on  sea  data.  A set  of  base  line  false 
alarm  rate  curves  are  described  in  Section  3.1,  while  Section 
3.2  discusses  the  actual  sea  data  false  alarm  rate  curves. 
Section  3.3  provides  a discussion  of  the  absolute  performance 
of  the  CAD  model  on  target  tracks.  This  performance  is  in 
turn  compared  with  the  performance  of  the  operator  on  both 
single  and  multiple  ping  events  in  Section  3.4. 

3.1  BASE  LINE  FALSE  ALARM  RATE  CURVES 

(U)  To  provide  a basis  for  comparison  of  CAD  effects  on 
reverberation,  a large  set  of  noise  was  generated  and  passed 
through  the  CAD  process.  Initially  4.2  x 10  independent 
samples  of  Gaussian  noise  were  obtained.  This  sequence  was 
treated  as  a sampled  time  sequence  with  a sampling  rate  of 
1,000  samples  per  second.  The  sampled  time  sequence  was 
bandlimited  by  the  application  of  a digital  5th  order 
Butterworth  filter  with  a 3 dB  points  at  100  and  200  Hz.  The 
bandlimited  noise  was  passed  through  a linear  rectifier 
followed  by  a 10  sample  finite-time  perfect  averager.  The 
resulting  "envelope"  sequence  was  divided  into  396  sections 
each  containing  10752  samples.  The  data  was  first  thresholded 
and  sorted  to  obtain  the  curve  of  false  alarm  rate  vs.  log 
likelihood  ratio  shown  in  Fig.  5.  Secondly,  the  data  was 
processed  through  the  CAD  model  and  sorted  to  obtain  the  curve 
shown  in  Fig.  6.  Each  of  these  curves  can  be  accurately 
represented  by  an  expression  of  the  form; 


FAR 


A+B-LfC-L^ 
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Figure  5.  False  Alarm  Rate  (Sec 
vs.  Logarithm  of  the  Likelihood 
Ratio  for  Noise  Without  CAD 
Processing 
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Figure  6.  False  Alarm  Rate  (Sec“^) 
vs.  Logarithm  of  the  Likelihood 
Ratio  for  Noise  after  CAD  Processing 
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where 

FAR  = False  alarm  rate,  SEC 
L = log  likelihood  ratio,  and 
A,  B,  and  C are  constants. 

The  values  of  A,  B,  and  C shown  in  Table  I were  determined  by 
a least  mean  square  fit  process.  The  results  obtained  by 
evaluating  Eq.  (1)  are  plotted  on  Figs.  5 and  6. 


TABLE  I 

FALSE  ALARM  RATE  COEFFICIENTS 

FOR  NOISE 

WITHOUT  CAD 

WITH  CAD 

A 

1.9289 

1.8818 

(C) 

B 

-0.5573 

-0.5985 

C 

-0.0315 

-0.0130 

(U) 

From  the  curves  shown  in  Figs. 

5 and  6 it  is  clear 

that 

the 

CAD  model  has  adverse  effects  on 

the  noise.  This 

result  is 

not  unexpected  and  is  more  than 

compensated  for  by 

the 

benefits  derived  from  tracking. 

3.2 

FALSE  ALARM  RATE  CURVES  OBTAINED  USING  SEA  DATA 

(C)  Using  data  from  approximately  1250  ping  cycles  of 
the  TECHEVAL  data  base  the  false  alarm  rate  curves  with  and 
without  CAD,  shown  in  Figs.  7 and  8,  were  generated.  This 
data  set  represents  approximately  10,000  seconds  of  sea  data 
background.  The  false  alarm  rate  coefficients  are  given  in 
Table  II.  By  comparing  the  curves  of  Figs.  7 and  8 to  the 
base  curves  (derived  from  noise)  in  Figs.  5 and  6 it  can  be 
seen  that  the  false  alarm  rates  are  slightly  higher  for  sea 
data  than  for  noise. 
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Figure  7.  False  Alarm  Rate  (Sec  ) vs 
Logarithm  of  the  Likelihood  Ratio  for 
Sea  Data  Without  CAD  Processing 
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Figure  8.  False  Alarm  Rate  (Sec”^)  vs, 
Logarithm  of  the  Likelihood  Ratio  for 
Sea  Data  After  CAD  Processing 
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(U)  In  Fig.  8 a sharp  cutoff  of  false  alarms  occurs 
at  a log  likelihood  ratio  value  of  14.  This  cutoff  is  caused 
by  the  data  processing  procedure  which  automatically  calls  any 
track  with  a larger  log  likelihood  ratio  a signal  track.  To 
compensate  for  this  artificial  cutoff,  the  curve  fitting 
procedure  weights  errors  (i.e.  deviations  between  the  curve  fit 
function  and  the  measured  false  alarm  rate  function)  with  the 
false  alarm  rate.  This  weighting  process  causes  the  artificial 
cutoff  of  the  measured  function  to  have  a very  minimal  effect 
on  the  fit  curve  since  the  false  alarm  rate  is  very  low  in 
the  region  affected  by  the  cutoff.  The  resulting  curve  fit 
function  should  be  more  valid  than  that  which  would  be  obtained 
without  imposing  a cutoff  on  the  measured  false  alarm  rate  curve. 


TABLE  II 

FALSE  ALARM  RATE  COEFFICIENTS  FOR  SEA  DATA 


WITHOUT  CAD 

WITH  CAD 

A 

1.2254 

1.0638 

(c) 

B 

-0.3761 

-0.3621 

C 

-0.0178 

-0.0107 

3.3 

ABSOLUTE 

PERFORMANCE  MEASURE 

(U)  The  performance  of  the  CAD  model  on  target  tracks  is 
condensed  to  a four  entry  track  information  package  as  described 
in  Section  2.2.2.  One  track  information  package  is  generated 
for  each  track  sequence  processed.  The  four  entries  in  the 
track  information  package  are, 

(U)  (1)  N,  the  number  of  pings  which  were  processed, 

(U)  (2)  LLR,  the  maximum  track  log  likelihood  ratio 

obtained  by  the  CAD  model. 
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(U)  (3)  S/iJ,  the  estimated  value  of  signal-to-noise 

for  the  N-ping  sequence  going  into  the  CAD 
mode 1 , 

(U)  (4)  S/lJ;MAX,  the  maximum  signal-to-noise  ratio 

which  occurred  in  the  N-ping  sequence  going 
into  the  CAD  model. 

(C)  By  combining  the  information  about  CAD  performeince 
on  target  tracks  with  the  false  alarm  rate  information  described 
in  Section  3.2  it  is  possible  to  obtain  an  absolute  measure  of 
the  total  model  performance.  Several  means  are  available  for 
displaying  this  measure  of  the  CAD  model  performance.  One 
technique  described  in  the  first  quarterly  report*  is  to 
generate  modified  ROC  curves.  This  technique  requires  a 
parameterization  of  the  input  signal-to-noise  ratio.  Since  we 
have  no  direct  control  over  the  input  signal-to-noise  ratio, 
the  required  parameterization  must  be  accomplished  somewhat 
arbitrarily  as  described  in  the  first  quarterly  report.  To 
avoid  this  arbitrary  parameterization,  a different  technique 
for  displaying  the  results  has  been  developed. 

(C)  For  each  track  information  package,  the  false  alarm 
rate  that  would  be  associated  with  a threshold  set  so  that  the 
track  sequence  would  be  detected  is  determined  by  using  the 
maximum  track  log  likelihood  ratio  to  index  the  false  alarm 
rate  curve  associated  with  the  output  of  the  CAD  model.  Using 
this  false  alarm  rate  and  the  estimated  value  of  signal-to-noise 
ratio,  a point  is  plotted  on  a grid  of  signal-to-noise  ratio 


♦Courts,  H.  R. , "Processing  Procedures  for  the  Validation  of  a 
Computer-Aided  Detection  Model"  (U) , UNITECH  68-016-C,  30  July 
1968,  (CONFIDENTIAL). 
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vs.  false  alarm  rate.  This  process  is  repeated  for  all  available 
track  information  packages  with  a common  value  of  N,  the  number 
of  pings  the  model  was  allowed  to  use,  to  obtain  a distribution 
of  points  on  the  grid.  These  points  are  fitted  with  a quadratic 
expression  to  obtain  an  estimate  of  input  signal-to-noise  ratio 
vs.  false  alarm  rate. 


(C)  The  results  of  this  process  are  shown  in  Figs.  9 
through  16,  where  the  number  of  pings  is  varied  parameterically . 
The  signal-to-noise  ratios  are  given  in  dB  on  the  vertical  scale 
and  the  logarithm  (to  the  base  10)  of  the  false  alarm  rate  (in 
Sec  is  given  on  the  horizontal  scale.  The  symbol  "X"  is 
used  to  represent  points  derived  directly  from  the  track 
information  packages  and  the  symbol  is  used  to  plot  the 

curve  fit  function.  Many  large  signal-to-noise  ratio  tracks 
yield  points  which  are  off  the  scale  of  the  graphs  shown  in 
Figs.  9 through  16.  Even  though  these  points  are  not  shown, 
they  are  included  in  determining  the  coefficients  of  the  curve  fit 
since  they  do  present  valid  information  about  trends. 


(C)  To  provide  some  insight  into  the  benefits  derived 
from  tracking  it  is  interesting  to  determine  what  the  performance 
would  have  been  had  the  CAD  model  not  been  used.  An  alternate 
detection  procedure  would  be  to  use  the  largest  signal  occurring 
in  a track  for  detection,  making  no  attempt  to  integrate  along 
the  track.  If  this  were  done,  the  appropriate  false  alarm  rate 
curve  would  be  the  curve  given  in  Fig.  7 without  the  CAD  model. 
It  is  important  to  note  that  when  several  observations  are 
allowed  (such  as  6)  the  expected  value  of  the  largest  signal 
may  be  considerably  larger  than  the  average  signal  level.  To 
obtain  a display  of  the  performance  associated  with  this  type 
of  processing,  the  maximum  signal-to-noise  ratio  which  occurred 
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was  transformed  into  a log  likelihood  ratio  and  used  to  index 
the  false  alarm  rate  curve  without  CAD  to  determine  a false 
alarm  rate.  The  resulting  (FAR,  S/tl)  pairs  were  processed  as 
described  above  to  obtain  the  results  shown  in  Figs.  17  through 
24.  By  comparing  the  results  using  CAD  for  detection  with 
the  corresponding  results  using  the  peak  for  detection,  two 
conclusions  can  be  drawn; 

(C)  (1)  In  an  average  sense  the  detection  performance 

obtained  with  the  CAD  model  is  about  1 to  2 dB  better  than 
detection  performance  obtained  using  the  peak  signal  for 
detection  in  the  low  false  alarm  rate  region. 

(C)  (2)  The  detection  performance  obtained  with  the 

CAD  model  is  much  more  consistent  than  that  obtained  using  the 
peak  signal  for  detection. 

3.4  COMPARISON  OF  CAD  PERFORMANCE  WITH  OPERATOR  PERFORMANCE 

(U)  In  addition  to  the  absolute  measure  of  CAD  performance 
described  in  Section  3.3,  it  is  desirable  to  obtain  a performance 
comparison  relative  to  the  sonar  operator.  Section  3.4.1  provides 
a single  ping  performance  comparison  while  Section  3.4.2  provides 
multiple  ping  performance  comparison. 

3.4.1  Single  Ping  Comparison  of  CAD  Performance  with 

Operator  Performance 

(C)  As  described  in  Section  2.1  the  operator  responses 
were  logged  for  each  ping  cycle  in  the  TECHEVAL  data.  The 
possible  responses  were:  N-No  Signal,  W-Weak,  M-Medium,  and 

S-Strong.  These  four  response  levels  provide  information  about 
three  operator  threshold  levels,  i.e.  Strong,  Medium  or  Better 
and  Weak  or  Better.  By  combining  the  operator  response  data  with 
the  measured  signal  log  likelihood  ratios,  three  curves  of 
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probability  of  detection  vs.  log  likelihood  ratio  are  obtained. 
The  three  curves  obtained  from  processing  approximately  1200 
operator  responses  are  given  in  Fig.  25. 

(C)  The  extreme  variability  of  the  operator  response  is 
especially  evident  in  the  distribution  of  strong  signal  calls. 
Some  signals  with  a signal- to-noise  ratio  as  low  as  13  dB  were 
called  and  other  signals  with  a signal-to-noise  ratio  as 
large  as  22  dB  were  not  called  strong.  In  processing  the 
operator  response  data  it  appeared  that  the  responses  were  very 
consistent  in  a relative  sense  within  a run  but  not  consistent 
in  an  absolute  sense  from  one  run  to  the  next.  This  tendency 
toward  relative  rating  of  signals  within  a run  may  have  occurred 
intentionally  in  the  data  gathering,  in  which  case  the  summary 
results  presented  in  Fig.  25  do  not  represent  a good  measure 
of  the  operator's  ability  to  perform  an  absolute  rating. 

(U)  It  is  possible  to  associate  a false  alarm  rate  with 
each  of  the  operator  response  threshold  levels  through  the 
relation, 


■]  * d-t  , 


where 


FAR(-t)  is  the  single  ping  false  alarm  rate  function 
described  earlier,  and 

Pjj(-t)  is  one  of  the  three  curves  describing  operator 
performance. 
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Fig.  25  OPERATOR  SINGLE  PING  PERFORMANCE  RESULTS 


CONFIDENTIAL 


dFAR  ( /.) 

In  words,  — ^ • dt  is  the  differential  rate  at  which  noise 
at 

events  occur  with  log  likelihood  ratios  between  t and  t + dt,  and 
Pjj  is  the  probability  that  the  noise  event  will  be  called  when 
it  does  occur. 

(U)  The  false  alarm  rates  associated  with  the  three 
operator  response  thresholds  may  be  compared  with  the  false 
alarm  rates  associated  with  a perfect  threshold  device  set  to 
obtain  50%  probability  of  detection  at  the  same  log  likelihood 
value  achieved  by  the  operator.  This  approach  provides  the 
desired  single  ping  comparison  of  the  CAD  model  with  the 
operator  in  terms  of  a difference  in  false  alarm  rate  at  the  same 
probability  of  detection.  These  results  are  summarized  in 
Table  III. 

TABLE  III 

SUMMARY  OF  OPERATOR  RESPONSES 


Response 

Threshold 

Operator 
False  Alarm 
Rate 

50%  Detection 
Level 

Threshold  Detector 
False  Alarm  Rate 

(FftR)^p 

(FMUth 

Weak 

1.30 

LLR=2,S/lJ=9.3  dB 

1.5 

0.8666 

Medium 

0.290 

LLR=8,S/N=14. 5 dB 

0.06 

4.8333 

Strong 

0.0276 

LLR=13,S/N=17.4  dB 

0.0015 

18.40 

3.4.2 

Multiple  Pina 

Comparison  of  CAD 

Performance  with 

Operator  Performance 


(C)  From  the  CAD  performance  curves  described  in  Section 
3.3  a curve  of  decrease  in  signal-to-noise  ratio  required  for 
50%  probability  of  detection  (from  that  required  when  N=l)  vs. 
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number  of  pings  (N)  of  integration  was  generated.  This  curve 
is  shown  in  Fig.  26.  Curves  of  10  log  N,  7 log  N,  and  5 log  N 
are  included  for  reference.  The  curve  labeled  "OPERATOR"  was 
obtained  from  Fig.  3-53  of  the  AN/SQS-26CX  Sonar  Signal  Processing 
Review. * 


\ 


Sonar  Signal  Physics  Committee,  "AN/SQS-26CX  Sonar  Signal 
Processing  Review,"  (U) , 1 March  1966,  (CONFIDENTIAL). 
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APPENDIX  A.  DESCRIPTION  OF  THE  CAD  MODEL 
A.l  THE  CAD  MODEL 

This  appendix  is  intended  to  provide  some  insight  into 
the  computer-aided  detection  model. 

The  logical  flow  of  data  within  the  CAD  model  is  shown 
in  Fig.  Al.  The  natural  time  base  for  an  active  track  detection 
model  is  the  ping  interval,  depicted  in  Fig.  Al  by  the  layers 
labeled  echo  cycles  N-1,  N,  and  N+1.  The  CAD  model  uses  two 
different  types  of  input  information,  that  stored  in  a master 
status  file  and  that  coming  from  the  sonar  signal  processor. 

Information  pertaining  to  possible  target  tracks  is 
stored  in  the  master  status  file  in  the  form  of  multiple  ping 
event  packages.  Each  of  these  packages  consists  of  data  which 
was  obtained  from  one  or  more  ping  cycles  and  which  shows 
promise  of  defining  a target  track.  During  echo  cycle  N,  single 
ping  event  packages  are  formed  from  ping  N and  used  as  input  to 
the  model.  When  viewed  in  this  way  it  is  seen  that  the  goal  of 
the  model  is  to  combine  single  ping  event  packages  from  each  echo 
cycle  in  an  optimal  way  with  multiple  ping  event  packages  from  the 
master  status  file  to  produce  an  updated  master  status  file.  The 
information  in  the  updated  master  status  file  is  then  passed  to 
the  next  echo  cycle  for  use  as  input  by  the  track  detection  model 
during  that  echo  cycle. 

The  track  strength  of  each  possible  track  represented 
in  the  status  file  can,  during  any  echo  cycle,  be  tested  against 
a threshold,  and  if  the  threshold  is  exceeded  the  information  in 
the  multiple  ping  event  package  can  be  used  to  drive  a display. 
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Fig.  A-l  LOGICAL  FLOW  OF  DATA 
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A. 2 EVENT  PACKAGES 

A. 2.1  Single  Ping  Event  Packages 

The  main  function  of  the  CAD  model  is  to  combine  the 
single-ping  information  received  from  a sonar  signal  processing 
unit  with  the  information  from  previous  pings  to  produce  an 
updated  master  status  file.  The  first  step  in  this  process  is 
the  conversion  of  processor  output  into  single-ping  event 
packages . 

Information  received  during  an  echo  ranging  cycle  is 
divided  into  single  ping  event  packages  by  the  application  of  a 
threshold  to  the  output  of  the  sonar  signal  processing  unit.  It 
is  often  possible  to  pick  a threshold  sufficiently  low  to  pass 
any  useful  signal  and  still  sufficiently  high  that  95-99%  of  the 
noise  will  not  exceed  the  threshold.  The  single  ping  event 
packages  contain  the  range  position  of  the  event  and  the  single 
ping  log  likelihood  ratio  associated  with  the  event. 

The  likelihood  ratio,  as  applied  to  single  events,  is 
defined  as  the  ratio  of  the  probability  that  the  event  peak  is  a 
result  of  signal  divided  by  the  probability  that  it  is  a result 
of  noise.  The  specific  transformation  from  peak  height  to  log 
likelihood  ratio  is  dependent  on  the  type  of  signal  processing 
used  and  the  average  signal- to-noise  ratio  expected  when  a signal 
is  present.  Often  the  exact  equation  for  transformation  from 
peak  height  to  log  likelihood  ratio  is  quite  complicated.  The 
transformation  equations  for  a replica  correlator  and  for  the 
statistical  wave  period  processor  have  been  investigated  in  some 
detail,  however,  and  it  has  been  found  that  in  the  area  of 
interest  a linear  transformation  from  peak  height  to  log  likeli- 
hood ratio  is  very  accurate. 
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As  mentioned  above,  the  specific  transformation  also 
depends  on  the  signal- to-noise  ratio  expected  when  a signal  is 
present.  To  specify  this  quantity  the  minimum  signal- to-noise 
ratio  which  will  permit  detection  in  a reasonably  small  number 
of  echo  cycles  is  used.  In  this  way  the  system  is  optimum  for 
signals  near  the  minimum  detectable  level.  The  derivation  of 
the  transformation  from  peak  height  to  log  likelihood  ratio  is 
presented  in  Appendix  B. 

A. 2.2  Multiple  Ping  Event  Packages 

As  mentioned  above,  information  pertaining  to  possible 
target  tracks  is  maintained  in  the  master  status  file.  It  is 
important  to  note  that  the  storage  required  does  not  depend  on 
the  duration  of  a target  track  but  only  on  the  number  of  possible 
tracks  under  investigation  at  a given  time.  The  number  of  tracks 
under  investigation  is  controlled  by  a retention  threshold  employed 
in  the  main  processing  loop  of  the  CAD  model.  A possible  target 
track  is  defined  in  computer  memory  as  long  as  its  track  likelihood 
ratio  exceeds  the  retention  threshold.  (The  likelihood  ratio  is 
the  ratio  of  the  probability  that  the  events  which  compose  the 
track  represent  signal  divided  by  the  probability  that  they  represent 
noise.)  In  the  validation  study  a likelihood  ratio  of  3 was  used 
as  the  retention  threshold. 

A possible  target  track  is  defined  by  a multiple  ping 
event  package  in  the  master  status  file.  Each  package  contains 
the  following  four  functional  quantities  used  to  describe  a 
possible  target  track: 

(1)  Logarithm  of  the  track  likelihood  ratio,  TLLR. 

(2)  Range  position  of  the  latest  processor  output 
peak  to  contribute  to  the  track,  LR. 
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(3)  Expected  range  position  in  the  next  ping  of  a 
peak  belonging  to  the  track,  ER. 

(4)  Previous  ping  bit  sequence. 

The  track  likelihood  ratio  provides  a measure  of  the 
track  strength.  To  avoid  the  multiplication  involved  in  track 
likelihood  ratio  computations,  it  is  convenient  to  work  with  the 
logarithm  of  the  likelihood  ratio.  When  a new  signal  is  received 
and  linked  with  a track,  the  track  likelihood  ratio  is  updated 
by  multiplying  the  old  track  likelihood  (that  of  the  multiple 
ping  event)  by  the  single  signal  likelihood  ratio  (that  of  the 
single  ping  event) , then  dividing  by  a loss  factor  dependent  upon 
the  deviation  of  the  single  signal's  range  position  from  the 
range  position  expected  for  a track  sample. 

The  expected  range  position  of  the  track  on  a given  ping 
is  determined  by  linear  extrapolation  from  the  position  of  the 
track  peaks  on  the  previous  two  pings.  The  expected  range 
position  and  the  previous  range  position  of  a track  can  be  combined 
to  produce  an  estimate  of  target  range  rate. 

The  previous  ping  bit  sequence  is  a string  of  24  bits. 

Each  bit  represents  a previous  ping  and  is  set  only  if  that  ping 
contains  a signal  peak  which  belonged  to  the  target  track. 

A .3  MACTER  STATUS  FILE  UPDATING 

The  next  stage  in  the  combination  of  single-ping 
information  with  multiple-ping  events  is  to  update  the  master 
status  file  and  thus  to  produce  a new  set  of  multiple-ping  events 
for  future  pings. 

Updated  entries  are  made  in  the  master  status  file  in 
three  functionally  different  ways.  These  are; 

(a)  Large  single  ping  events, 

(b)  Single  ping  events  which  are  linked  with  multiple 
ping  events,  and 
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(c)  Large  multiple  ping  events  which  are  not  linked  to 
new  peaks. 

The  procedures  used  for  making  each  type  of  entry  are 
described  below. 

A. 3.1  Large  Single  Ping  Events 


If  the  log  likelihood  ratio  of  a single  ping  event 
exceeds  the  retention  threshold,  then  a single  ping  entry  to  the 
master  status  file  will  usually  be  made.  This  single  ping  entry 
to  the  master  status  file  will  be  blocked  if  the  peak  is  linked  in 
a very  consistent  way  with  an  existing  track.  This  blocking 
procedure  is  described  under  processing  of  linked  events.  On  the 
first  echo  ranging  cycle,  only  single  ping  entries  are  made. 

The  expected  range  position  is  set  equal  to  the  single  event 
range.  The  log  of  the  track  likelihood  ratio  is  set  equal  to 
the  single  ping  log  likelihood  ratio. 


A. 3. 2 


Linked  Events  and  Large  Multiple  Ping  Events  Which 


Are  Not  Linked  to  New  Peaks 


When  a single-ping  event  occurs  in  a range  position  in 
the  vicinity  of  the  expected  position  of  a target  track,  the 
possibility  of  a "link"  between  the  single- ping  event  and  the 
multiple  ping  track  entry  arises.  The  linking  process  is  the  most 
essential  part  of  the  target  track  detection  model  since  it  is  the 
vehicle  for  achieving  ping-to-ping  integration  along  a track. 

The  logic  used  to  accomplish  the  linking  of  tracks  with 
new  peaks  is  shown  in  Fig.  A2.  Each  block  on  the  flow  chart  shown 
is  numbered  and  the  following  numbered  paragraphs  provide 
functional  descriptions  of  the  corresponding  flow  chart  blocks. 
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Fig.  h-2  COMPUTER  AIDED  DETECTION  FLOWCHART 
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(1)  The  history  table  is  scanned  to  locate  the 
history  packet  with  the  largest  log  likelihood  ratio.  Further 
processing  described  below  will  be  directed  toward  linking  the 
entry  located  with  a peak  in  the  current  echo  cycle  or  propagating 
the  entry  if  no  link  is  achieved. 

(2)  If  the  TLLR  of  the  entry  located  in  step  (1)  is 
zero,  then  all  history  packet  entries  have  been  processed  and 
control  is  transferred  to  the  routine  to  process  single  peaks  as 
described  in  Section  A. 3.1  above.  If  the  TLLR  is  zero,  control 
passes  on  to  step  (3)  below. 

(3)  This  block  of  logic  determines  the  minimum  and 
maximum  r.anges  applicable  for  linkage  with  a peak,  based  on  the 
maximum  range  rate  to  be  tracked.  A quantity,  K,  is  set  equal  to 
the  number  of  preceding  missed  pings  + 1.  The  maximum  range 
change  to  be  allowed,  MAX.  A,  is  calculated  by  multiplying  a 
basic  maximum  range  change  by  K.  This  process  effectively  opens 
the  range  gate  when  missed  pings  occur.  The  minimum  and  maximum 
ranges  to  be  allowed  are  calculated  from  the  last  range  measured 
by  subtracting  and  adding  MAX.  A»  respectively. 

(4)  A test  is  made  to  determine  whether  a range  rate 
estimate  is  available.  If  the  track  under  consideration  has 
only  one  ping  with  a detection,  then  no  range  rate  estimate 

is  available  and  the  logic  proceeds  to  step  (8) . Otherwise  it 
proceeds  to  step  (5) . 

(5)  This  calculation  determines  the  maximum  deviation 
from  the  expected  position  that  still  permits  a link  to  be  made 
with  a positive  contribution  to  the  peak  log  likelihood  ratio 
from  the  link.  In  general,  when  a link  is  made  the  track  log 
likelihood  ratio  is  updated  by  an  equation  of  the  form. 
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where 


(TLLR)jj  = (TLLR)q  + PLLR  - D(1R-ER|),  (A1) 


(TLLR)jj  = the  new  track  log  likelihood  ratio, 

( TLLR) Q = the  old  track  log  likelihood  ratio, 

PLLR  = the  single  peak  log  likelihood  ratio,  and 
D(|R-ER|)  = some  deviation  loss  function  of  the  difference 
between  the  expected  range  and  the  measured 
range . 

The  maximum  deviation.  A,  is  defined  as  the  value  such 


that 


(TLLR)q  - D(a)  = 0 


(A2) 


The  algorithm  used  in  computing  D has  been  derived  by 
assuming  that  the  range  distribution  of  signal  peaks  belonging 
to  a track  relative  to  the  expected  range  is  Gaussian  with  a 
mean  of  zero  and  a standard  deviation  of  (K*a) . It  is  important 
to  note  that  the  algorithm  does  compensate  for  the  increase 
in  position  ambiguity  following  missed  pings.  The  actual 
algorithm  used  in  computing  D is 


D = - -tn  ( 


N 


K-a/2^ 


-)  + 


2(Ka) 


(A3) 


\ 
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where  N is  the  number  of  samples  in  a resolution  interval  at  the 
signal  processor  output  and  d is  the  deviation  fran  the 
expected  position.  By  combining  Eqs.  (Al) , (A2) , and  (A3) 
and  solving  for  A one  can  obtain 


A = /[TLLR  + -f.n(-^;,—  ) - .tn(K)  ] ^ 2 » (a-K)^  . (A4) 

a/2TT 

Analytic  methods  do  not  readily  yield  a value  for  a» 
but  inspection  of  typical  target  tracks  indicates  a value  of 
30  ms.  The  number  of  samples  in  a resolution  interval,  N,  is 
10  since  the  sampling  rate  used  is  1000  Hz  and  the  time 
resolution  of  the  sonar  signal  processor  is  10  ms.  Insertion  of 
these  coefficients  in  Eq.  (A4)  yields 


A = /LTLLR  ~ 2.02  7,n(K)  ) • 1800  • . (A5) 

(6) , (7)  The  purpose  of  this  logic  is  to  reset  the 
range  gate  limits,  if  necessary,  based  on  the  maximum  deviation 
that  can  be  tolerated  in  the  expected  range. 

(8)  The  initial  conditions  for  the  scan  of  peaks  linked 
to  a given  track  include: 

(a)  BB.SW=0,  This  switch  is  used  to  determine 
whether  or  not  a "BRICK-BAT"  link  has  been 
achieved  as  described  below. 
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(b)  Two  storage  control  addressr.3  are  set,  both 
initially  pointing  to  the  same  address.  STB 
will  remain  static,  pointing  to  the  first 
linked  event  package  to  be  formed.  STD  will 
advance  as  successive  event  packages  are 
formed,  when  more  than  one  peak  is  linked  to 
the  given  track. 

(c)  Necessary  loop  controls  are  set  for  a scan  of 
all  new  peaks. 

(9)  Each  peak  is  tested  to  be  within  the  allowed 
range  gate.  When  a peak  is  found  within  the  allowed  range  gate, 
control  passes  to  step  (10). 

(10)  The  next  step  after  locating  a peak  within  the 
allowed  range  gate  is  to  determine  the  position  deviation 

loss  associated  with  the  peak/track  pair.  If  the  track  only  has 
one  previous  "hit,"  then  no  range  rate  estimate  is  available  and 
control  passes  to  step  (11) . Otherwise  control  passes  to  step 
(12)  . 

(11)  When  making  the  first  link  of  a new  track  no 
range  rate  estimate  is  available,  so  the  standard  method  for 
calculating  the  position  deviation  loss  is  not  applicable.  In 
this  case  the  deviation  loss  applied  must  compensate  for  the 
ambiguity  introduced  in  scanning  the  range  gate  to  locate  the 
peak.  In  the  validation  study  the  standard  range  gate  is  240  ms. 
Since  the  resolution  of  the  sonar  is  10  ms  there  were  24 
independent  samples  in  the  240  ms  gate.  A deviation  loss  of 

/n  (24),  or  3.18,  is  applied  to  compensate  for  the  link  ambiguity. 
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(12)  When  a range  rate  estimate  is  available,  Eq.  (A3) 
is  used  to  calculate  the  deviation  loss  D.  As  before,  the 
coefficients  N=10  and  o=30  are  used.  When  these  coefficients 
are  inserted  in  Eq.  (A3)  one  obtains. 

.2 

D = j + /n(K)  + 2.02  . (A6) 

1800-K^ 

(13)  From  Eq.  (Al)  it  can  be  seen  that  when  a link  is 
made  the  track  log  likelihood  ratio  will  increase  by  an  amount 
PLLR-D.  If  this  quantity  is  negative  the  link  is  not  made  and 
control  is  returned  to  scan  the  next  peak. 

(14) , (15) , (16)  At  this  point  the  decision  to  make 
a linked  entry  is  complete.  The  purpose  of  the  logic  in  steps 
(14),  (15)  and  (16)  is  to  classify  the  link  as  a "BRICK-BAT" 
link  if  both  PLLR-D  and  TLLR-D  exceed  a fixed  level,  BB. LEVEL. 

The  intent  of  the  "BRICK-BAT"  link  is  to  assert  that  we  are  sure 
that  the  link  is  correct,  and  then  to  prevent  any  other 
interactions  of  the  track  package  and  the  peak  event.  The  reset 
of  STD  to  STB  will  cause  any  linked  event  packages  already  formed 
v/ith  this  track  to  be  discarded. 

(17)  to  (22)  These  steps  carry  out  the  necessary  operations 
to  form  a linked  event  entry  package  in  the  new  history  file. 

The  new  event  package  is  stored  in  the  file  at  a position  indicated 
by  STD.  The  new  track  log  likelihood  ratio  is  calculated  in 
step  (17)  using  Eq.  (Al) . A limit  is  applied  to  the  value  allowed 
for  the  track  log  likelihood  ratio  in  step  (18) . The  new  expected 
range,  ERN,  is  calculated  using  the  peak  range,  PR,  and  the  last 
range,  LR,  obtained  from  the  old  track  history  packet.  The 
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current  peak  range,  PR,  is  entered  in  the  new  history  packet  as 
the  new  last  range.  A 1-bit  is  entered  to  indicate  a hit  in  the 
hit/miss  string.  The  storage  control  pointer,  STD,  is  advanced 
in  step  ( 22) . 

(23)  A test  is  made  to  determine  whether  "BRICK-BAT" 
conditions  were  met  on  the  link  just  completed.  If  these 
conditions  were  not  met  control  is  returned  to  scan  the  next 
peak.  If  the  conditions  were  met  control  proceeds  to  step  (24)  . 

(24)  If  a "BRICK-BAT"  link  was  made  the  peak  log 
likelihood  ratio  is  cleared  to  prevent  the  peak  from  causing  a 
single  peak  entry. 

(25)  The  track  log  likelihood  ratio  in  the  old  history 
packet  is  cleared  to  zero  and  control  is  returned  to  scan  for  the 
next  track  to  be  processed. 

(26)  Control  reaches  this  point  when  all  peaks  have  been 
tested  for  a link  with  a given  track  and  a "BRICK-BAT"  link  has 
not  been  achieved.  If  no  links  were  made,  control  passes  to 

step  (28)  to  test  for  packet  propagation. 

(27)  If  several  links  were  made,  only  the  best  link  is 
retained.  The  choice  of  the  best  link  is  based  on  the  maximum 
TLLRN. 

(28)  Control  reaches  this  point  only  if  no  links  were 
made.  If  the  track  log  likelihood  ratio,  TLLR^  is  less  than  6, 
control  is  passed  to  step  (25)  and  the  track  is  dropped. 

(29)  If  the  number  of  successive  missed  pings  exceeds 
6,  control  passes  to  step  (25)  and  the  track  is  dropped. 


UNCLASSIFIED 


hor’o  l'3V'5 


UNCLASSIFIED 


62 


(30),  (31) , (32)  These  steps  carry  out  the  necessary 
operations  to  propagate  a track  package  from  the  old  history  file 
to  the  new  history  file.  The  track  log  likelihood  ratio  is 
reduced  by  3 to  reflect  the  effect  of  a missed  ping.  The 
expected  range  is  extrapolated  and  a 0-bit  is  entered  in  the 
hit/miss  string. 

A. 3. 3 Track  Redundancy  Removal 

The  procedures  described  in  Sections  A. 3.1  and  A. 3. 2 
above  allow  the  possibility  of  generating  redundant  entries 
in  the  new  history  file.  To  eliminate  redundancies,  the  new 
history  file  is  scanned  to  locate  entries  for  which  both  the 
last  range,  LR,  and  the  expected  range,  ER  are  the  same.  When 
several  of  these  entries  have  been  located  only  the  entry  with 
the  largest  track  log  likelihood  ratio  is  retained. 
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APPENDIX  B.  DERIVATION  OF  THE  TRANSFORMATION  FROM 
PEAK  HEIGHT  TO  LOG  LIKELIHOOD  RATIO 


The  transformation  from  processor  output  peak 
amplitude  to  logarithm  of  likelihood  ratio  is  the  initial 
processing  stage  of  the  target  tracking  model.  Model  performance 
is  directly  related  to  the  accuracy  of  the  transformation.  The 
validity  of  the  log  likelihood  conversion  algorithm  is  in  turn 
dependent  upon  an  accurate  statistical  description  of  processor 
output  amplitude  in  a multi-ping  signal-plus-noise  environment. 
This  Appendix  presents  two  different  statistical  approximations  to 
a multi-ping  signal-plus  noise  distribution  and  documents  their 
associated  likelihood  ratio  transformations. 

The  likelihood  ratio,  L(x),  associated  with  a processor 
output  peak  of  amplitude  x is  given  by 


P (x) 

" P^(x)  ' 

where  P^  is  the  probability  density  function  of  amplitude  in  a 
processor  output  exhibiting  a signal- to-noi se  ratio  S,  and  P^ 
is  the  probability  density  function  of  amplitude  in  noise-alone 
processor  output. 

One  immediate  problem  arises,  since  the  likelihood  ratio 
L(x)  defined  above  is  dependent  upon  the  average  signal- to-noise 
ratio  S assumed  at  the  processor  output.  In  order  to  implement 
the  likelihood  ratio  conversion,  the  value  of  S must  be  fixed  as 
a system  parameter.  Yet,  the  actual  value  of  S cannot  be  known 
before  the  fact,  and  it  may  vary  widely  from  one  contact  to 
another.  One  solution  to  this  problem  is  to  set  S in  the  range 
of  the  minimum  detectable  signal- to-noise  ratio  at  the  output 
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of  the  processor.  When  this  is  done,  likelihood  ratio  estimates 
computed  according  to  Eq.  (Bl)  will  be  inexact  only  for  signal- 
to-noise  ratios  somewhat  larger  or  smaller  than  S.  This 
inexactness  will  not  affect  overall  system  performance  because  the 
smaller  signals  are  presumably  non-detectable  at  the  processor 
output  and  the  larger  signals  require  less  processing  gain  for 
detectability.  Thus,  when  S is  set  at  a marginally  detectable 
signal- to-noise  ratio,  the  tracking  model  is  optimized  to  provide 
maximum  processing  gain  for  those  signals  that  are  only  barely 
detectable.  In  the  validation  study,  the  region  of  minimal 
detectable  signal- to-noise  ratio  at  the  output  of  the  correlator 
is  from  10  dB  to  12  dB. 


The  next  problem  in  deriving  a log  likelihood  trans- 
formation is  to  determine  the  probability  density  functions 
and  required  for  the  computation  of  Eq.  (Bl)  . Two  classes  of 
probability  density  functions  have  been  used  to  derive  log 
likelihood  ratio  conversion  algorithms.  The  first  class  of 
functions,  discussed  in  Section  B.I.,  arise  from  the  assumption 
that  the  ping-to-ping  signal-plus-noise  statistics  of  correlator 
output  amplitude  are  described  by  the  envelope  distribution  of 
an  ideal  signal  in  Gaussian  noise.  The  second  class  of  functions, 
discussed  in  Section  B.2.,  arise  from  the  assumption  that  signal- 
plus-noise  and  noise-alone  correlator  output  amplitude  statistics 
are  both  described  by  Rayleigh  distributions  with  different 
standard  deviations. 


B.l  LOG  LIKELIHOOD  RATIO  OF  IDEAL  SIGNAL  PLUS  GAUSSIAN 

NOISE  ENVELOPE 

If  the  assumption  is  made  that  signal  amplitude  is 
constant  from  ping-to-ping,  then  echo  returns  following  the 
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correlator  are  similar  to  short  CW  pulses  of  the  same  signal 
amplitude.  The  linear  rectifier  and  averager  following  the 
correlator  then  act  as  an  envelope  detector  for  these  pulses. 
Thus,  the  probability  density  functions  required  for  evaluating 
Eq.  (Bl)  are  those  which  describe  the  envelope  of  a sine  wave 
plus  random  noise.  These  density  functions  are  given  by 
S.  O.  Rice*  as 


Ps(X) 


„-S/N 
e e 


-X" 

2N 


Io(/2  /S/N 


P^(X) 


N ® 


2N 


where 

X is  the  height  of  the  envelope, 

N is  the  average  noise  power, 

S is  the  average  signal  power, 

is  the  modified  Bessel  function  of  order  0. 

Using  these  two  equations  to  compute  likelihood  ratio, 
Eq.  (Bl)  yields 


L(x)  - e I^(/2  /S/n  ^ ) 


*Rice,  S.  O.,  "Mathematical  Analysis  of  Random  Noise,"  Bell 
System  Technical  Journal,  Vol . 24,  1945,  p.  100. 
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As  given  by  Rice,  the  output  noise  mean,  u,  is  >/K t/2  and  the 
output  noise  standard  deviation  a is  /N( 2 - v/2) . If  r 
denotes  the  envelope  height  in  units  of  a relative  to  u,  then 

r = (X  - u)/a  , 

or  X = a • r + u 

= /n(2  - tt/2)  • r + /Ntt/2  . 

Substitution  for  X in  the  equation  for  L(x)  above  yields 


L(x)  = [v/S/N  (/4-tt  • r + /rr)  1 . (B2) 

The  conversion  algorithm  for  correlator  output  amplitude 
samples  X is  then  obtained  by  taking  the  natural  logarithm  of 
Eq.  (B2)  where  X has  been  converted  to  r using  measured  values  of 
U and  CT.  This  has  been  done  in  Fig.  B-1,  where  the  logarithm 
of  Eq.  (B2)  has  been  plotted  as  a function  of  r for  three  values 
of  S/N  that  yield  average  output  signal- to-noise  ratios  of  10  dB, 

11  dB  and  12  dB. 

The  curves  that  appear  in  Fig.  B-1  thus  yield  the 
fruitful  result  that  the  logarithm  of  Eq.  (B2)  is  closely 
approximated  by  a straight  line  in  the  region  of  interest 
(i.e.  values  of  r from  2 to  6 which  correspond  to  output  signal- 
to-noise  ratios  from  6 dB  to  15.5  dB) . The  logarithm  of 
likelihood  ratio  conversion  of  a measured  output  peak  of 
cimplitude  X then  reduces  to 
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In  L(x)  = a • r + b 


= a(-^)  + b 
a 


■tn  L(x)  = 


- . X + b - , 

o a 


where  a and  b are  the  slope  and  slope  intercept  values  defining 
the  appropriate  straight  line  in  Fig.  B-1  and  u and  a are 
measured  values  of  correlator  output  noise  mean  and  standard 
deviation.  For  example,  if  the  expected  correlator  output 
signal- to-noise  ratio  is  to  be  set  at  10  dB,  then  Fig.  B-1  is 
examined  and  values  of  a = 2.0  and  b = -2.86  are  computed  as 
the  values  determining  the  straight  line  marked  10  dB.  In 
this  case,  then,  the  transformation  from  peak  height  X to 
logarithm  of  likelihood  ratio  in  L(x)  is  given  by 


<,n  L(x)  = 2.0  - - 2.86  - 

a O’ 

In  summary,  the  assumption  that  correlator  output 
peak  statistics  are  described  on  a multi-ping  basis  by  the 
distribution  of  constant  amplitude  ideal  signal  plus  Gaussian 
noise  yields  the  easily  implemented  linear  transformation  in 
Eq.  (B3)  from  peak  height  to  logarithm  of  likelihood  ratio. 

B.2  LOG  LIKELIHOOD  RATIO  OF  RAYLEIGH  SIGNAL  PLUS  RAYLEIGH 

NOI SE 

Although  the  log  likelihood  ratio  conversion  discussed 
in  Section  B.l  is  simple  to  implement,  the  assumptions  upon  which 
it  is  based  seem  rather  stringent.  In  particular,  the  assumption 
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that  consecutive  echo  cycles  contain  ideal  constant  amplitude 
signals  plus  Gaussian  noise  is  not  confirmed  by  empirical  data, 
which  indicates  that  peak  heights  have  a larger  variation  than 
that  expected  in  an  ideal  case.  For  this  reason,  a "noisy" 
signal  distribution  was  considered.  This  distribution  is  to  be 
denoted  as  Rayleigh  signal  plus  Rayleigh  noise. 


The  Rayleigh  probability  density  function,  which  is 
identical  to  the  Rice  noise-alone  distribution,  is  given  by 


P(X)  = e 2a" 
a 


The  signal-plus-noise  hypothesis  investigated  in  this  section 
assumes  that  both  signal-plus-noise  and  noise-alone  peaks  are 
distributed  by  Rayleigh  density  functions  Pg  and  Pj^  with 
different  parameters  and  Hence, 


Pg(X) 


and 


Pu(S) 


X2 

2a\ 


Under  the  assumption  of  the  above  density  factors,  the 
likelihood  ratio  measurement  L(X)  is  given  by 


L(X) 


X2(^ 

2a 


N 


2 a: 
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The  mean  values  u and  n and  standard  deviation  values 

n s 

and  of  the  signal  and  noise  distributions  are  given  by 


= % 

/TT/i2  , 

II 

p 

D 

/tt/2  , 

II 

P 

CR 

v/2-  n/2 

/2-  tt/2 

Consequently,  amplitude  values  X,  when  measured  in  units  r of 

noise  standard  deviation  o relative  to  the  noise  mean  □ are 

n n 

given  by 


X = On  r + Un 


= /2-TT/2  ’ 

Substituting  the  above  for  X in  the  previous  expression  for  L(X) 
the  following  is  obtained: 

^ -i,\  n ns 

L(X)  = — ^ e 

s 


^ 

n 20 , 

— e £ 

2 
a 

s 
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where  = / 2- v/’l  and  k2  = /tt/2 

Finally,  taking  log  likelihood  ratio,  we  obtain 


a a 

In  L(X)  = in(-^)  + (4  - -’V)  [l^,r+k  ] 
a ^ ^ 2a  ^ 

s 


(B4) 


0 

The  relation  between  — and  correlator  output  signal- to-noise 

ratio  is  easily  obtained.  For  example,  since  the  tracking 
model  is  to  be  optimized  for  10  dB,  it  is  required  that 


Consequently, 


10  = 20  log  ( 


^s-^n, 


a k„-a„  k_ 
s 2 n 2, 


= 20  log  ( ° " - ) 

°n^l 

a k-  k_ 

= 20  log  ^ ) 

“n  ’^l  ^1 


a k,  a 

^ /To  + 1 = 2.65  and  ^ = 0, 


n ^^2 


377 


Thus,  to  optimize  Eq.  (B4)  for  a 10  dB  correlator  output  signal- 

to-noise  ratio,  the  value  0.377  is  substituted  in  Eq.  (B4)  for 
a 

— and  the  following  conversion  algorithm  results; 

°^s 
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In  L(X)  = /.n( 0.377)  ^ - 0.377^)  [ r + ^ 


= - 1.275  + (0.706)r  + (0.185)r' 


Equation  (B4)  has  been  evaluated  for  values  of  — 

Q 

corresponding  to  correlator  output  signal- to- noise  ratios 


of  10  dB  and  11.5  dB.  The  resulting  curves,  in  which  ^n  L(X)  is 
plotted  as  a function  of  r,  appear  in  Fig.  B-2.  The  quadratic 
transformation  indicated  in  Eq.  (B4)  may  be  simplified  further  by 
computing  a least  mean  square  linear  fit  in  the  region  cf  interest 
to  the  curves  in  Fig.  B-2.  This  has  been  done  for  the  10  dB 
curve  and  the  resulting  linear  fit  is  indicated  by  the  dotted 
line.  The  slope  and  slope  intercept  values  a and  b yield  a simpli 
log  likelihood  transformation  given  by 


f ied 


in  L(X)  = 2.45r  - 5.2 


In  summary,  the  assumption  of  a Rayleigh  signal  plus 
Rayleigh  noise  distribution  yields  a quadratic  transformation  from 
peak  height  X to  log  likelihood  ratio  that  is  quite  practical  to 
implement  and  which  further  may  be  approximated  by  a linear 
transformation  in  the  range  of  interesting  signal- to-noise  ratio. 
This  form  of  log  likelihood  ratio  transform  is  being  used  in  the 
validation  study. 
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APPENDIX  C.  A SIMPLIFIED  MODEL  FOR  PREDICTING  CAD  PERFORMANCE 


To  provide  some  insight  into  the  expected  pen^agmance 
of  the  computer-aided  detection  model,  it  is  useful  to  make  some 
simplifying  assumptions  about  the  data.  The  assumptions  to  be 
made  are : 

(1)  The  peaks  of  a signal  track  will  be  assumed  to  be 
equal  to  the  same  value  on  each  ping. 

(2)  The  deviation  of  the  peak  position  from  the 
expected  position  will  be  assumed  to  have  a standard  deviation 
which  is  equal  to  the  standard  deviation  used  in  calculating 
the  position  deviation  loss. 

When  a peak  is  linked  to  a track  the  track  log  likelihood  ratio 
is  updated  by  an  equation  of  the  form, 

TLLRN  = TLLRO  + PLLR  - D{d)  , (Cl) 


where 

TLLRN  = the  new  track  log  likelihood  ratio, 

TLLRO  = the  old  track  log  likelihood  ratio, 

PLLR  = the  log  likelihood  ratio  of  the  peak, 

D(d)  = a subtractive  loss  which  is  a function  of 
the  deviation  of  the  peak  position  from 
the  expected  position. 

The  subtractive  loss  function,  D,  compensates  for  the 
ambiguity  which  must  be  allowed  to  make  the  link.  The  evaluation 
of  the  D function  depends  on  whether  or  not  a range  rate  estimate 
is  available.  In  the  first  link  of  a new  track  no  range  rate 
estimate  is  available,  and  the  D function  is  evaluated  as  the  log 
of  the  number  of  independent  samples  in  the  allowed  link  range 
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gate.  In  the  validation  study  the  allowed  link  range  gate  is  240  ms 
and  the  resolution  of  the  sonar  is  ^(24) =3. 18.  when  a range  rate 
estimate  is  available  the  D function  is  evaluated  as 


where 


D(d) 


•tn( 


o/2v 

N 


+ 1) 


(C2) 


a the  standard  deviation  assumed  for  the  difference 
between  the  expected  range  and  measured  range 
for  signals.* 

N = the  number  of  independent  samples  in  one 
resolution  cell  at  the  sonar  output, 

d - the  difference  between  the  expected  range  and  the 
measured  range. 

Using  assumption  (2)  Eq.  (C2)  reduces  to 


D = 


-t-n( 


a/?iT 

N 


+ 1) 


(C3) 


Now  it  is  possible  to  write  an  expression  giving  the 
expected  track  log  likelihood  ratio,  TLLR,  as  a function  of  the 
number  of  pings,  K,  and  the  peak  log  likelihood  ratio,  PLLR: 


*^the  implantation  of  the  CAD  model  the  approximatic 

N ^ ^ jj-  has  been  used.  The  more  exact  form  has 

thJt''?h'^  parameter  variations  over  a rangt 

that  the  approximation  is  not  valid.  ^ 


been 

such 
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TLLR(PLLR,K)  = PLLR 


TLLR(PLLR,K)  = K • PLLR  - 3 . 18  - (K-2)  ( In  ( +1)  + -|) 


= K 


(PLLR  - + 1)  - ^)  + 2 


‘in  { 


g/?TT 


N 


+ 1)  - 2.18 


J 


From  Eq.  (C4)  one  can  see  that  the  expected  track  log 
likelihood  ratio  is  a linear  function  of  the  number  of  pings,  K, 
when  K is  greater  than  one.  It  is  important  to  note  that  the 
expected  track  log  likelihood  will  increase  with  K only  if  the 


peak  log  likelihood  ratio  is  greater  than  -t-n  ( ~ + 1)  + 


This 


implies  that  the  tracking  ambiguity  establishes  a floor  on  PLLR 
such  that  signals  of  lower  level  cannot  be  detected  even  with  an 
infinite  observation  time. 

In  the  validation  study  the  values  used  for  a and  N 
were  30  and  10  ms  respectively.  Figure  C-1  shows  curves  of  expected 
track  log  likelihood  ratio  vs.  number  of  pings  with  the  value  of 
average  peak  log  likelihood  ratio  parameterized.  As  shown  in 
Appendix  B the  peak  log  likelihood  is  directly  related  to 
signal- to- noise  ratio.  The  values  of  signal- to-noise  ratio 
equivalent  to  the  log  likelihood  values  are  also  given  in  Fig. 

C-1.  Note  that  the  range  of  signal- to-noise  ratio  from  10  dB  to 
13  dB  covers  the  "interesting"  range  of  operation.  That  is,  signals 
below  10  dB  cannot  be  detected  and  signals  above  13  dB  will  be 
detected  very  easily. 
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Figure  C-2  shows  a set  of  curves  which  are  functionally 
identical  to  those  of  Fig.  C-1  with  a = 10  ms.  The  lowest  detectable 
level  is  decreased  to  9.1  dB  by  the  improved  track  consistency. 

Next,  it  is  interesting  to  consider  how  the  false  alarm 
rate  curves  may  be  expected  to  change  as  the  standard  deviation 
assumed  for  the  difference  between  the  expected  range  and 
measured  range,  a,  is  changed.  Changing  this  parameter  has  an 
impact  on  the  model  as  indicated  in  Eq.  (C2) . The  intent  of  the 
deviation  loss  function,  D,  is  to  provide  a normalized  system  such 
that  we  may  vary  o without  changing  the  false  alarm  characteristics. 

large  set  of  noise  data  was  processed  through  the  model  using 
values  for  n of  30  ms  and  10  ms.  The  resulting  false  alarm  rate 
curves  are  shown  in  Figs.  C-3  and  C-4.  The  close  agreement  between 
the  two  curves  indicates  that  the  deviation  loss  function  as  given 
in  Eq.  (C2)  does  achieve  a normalized  system. 

By  combining  the  curves  of  Fig.  C-1  with  the  false  alarm 
rate  curve  given  in  Fig.  C-3  it  is  possible  to  obtain  a set  of 
curves  of  required  signal- to-noise  ratio  vs.  false  alarm  rate  with 
the  number  of  pings  parameterized.  These  curves  are  shown  in  Fig. 
C-5.  These  curves  may  be  compared  directly  with  the  measured  curves 
obtained  using  sea  data  (Figs.  10  through  14  in  Section  3.3) . 
Considering  the  simplicity  of  the  model  used  to  obtain  the  curve  of 
Fig,  C-5,  the  agreement  with  the  measured  results  is  excellent,  with 
differences  between  the  two  sets  of  curves  on  the  order  of  dB. 
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Figure  C-3.  False  Alarm  Characteristics 
of  the  CAD  Model  Using  a Standard 
Deviation  of  30  ms 
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SUMMARY  OF  SIMPLIFIED  MODEL  PERFORMANCE 
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APPENDIX  D.  SOME  EXAMPLE  DISPLAYS 

In  this  appendix  several  sets  of  displays  obtained  from 
sea  data  are  presented  to  give  a visual  comparison  of  displays 
with  and  without  CAD  processing.  Each  set  consists  of  four 
displays  which  will  be  referred  to  as  types  A,  B,  C,  and  D. 

Each  type  of  display  is  described  below. 

Type  A Display 

The  type  A display  provides  signal  and  noise  information 
without  CAD  processing.  The  display  is  formatted  with  one  line 
of  print  for  each  ping.  Horizontal  position  within  the  display 
is  proportional  to  time  within  the  ping  cycle  relative  to  the 
start  of  a range  gate.  The  digits  printed  indicate  dB  in  excess 
of  some  given  signal- to-noise  ratio  value  provided  in  the  header 
at  the  top  of  the  display.  The  header  also  provides  the  start  time 
of  the  range  gate  relative  to  the  transmit  pulse  in  ms,  and  the 
range  bin  width  in  ms. 

Type  B Display 

The  type  B display  provides  a tabular  representation 
of  information  about  the  target  track  selected  by  the  automatic 
track  localization  process.  The  first  column  is  the  ping  number. 
The  second  column  is  arrival  time,  relative  to  transmit  time,  of 
the  peak  selected  in  seconds.  The  third  column  is  the  signal- 
to-noise  ratio  of  the  peak  in  dB.  The  fourth  column  is  the 
deviation  of  the  peak  position  from  a least  mean  square  quadratic 
fit  to  the  values  in  column  two,  in  ms.  At  the  bottom  of  the 
table  the  standard  deviation  of  the  values  in  column  four  is  given 
to  provide  a measure  of  track  position  consistency. 
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The  type  C display  is  functionally  like  the  type  A 
display,  but  only  the  points  of  the  selected  target  track  are 
given . 


The  type  D display  is  similar  in  format  to  display 
types  A and  C.  The  results  displayed  are  the  signal  plus  noise 
output  of  the  CAD  model.  A limit  of  7 exists  for  this  display  as 
a result  of  the  limit  on  maximum  log  likelihood  ratio  imposed  in 
the  CAD  model.  Only  pings  with  a signal  present  are  displayed. 
Pings  with  a missed  signal  will  be  blank  even  though  the  CAD 
model  has  not  discarded  the  track.  A comparison  of  display  types 
A and  D provides  a good  visual  indication  of  the  impact  of  the 
CAD  model. 


UNCLASSIFIED 


CONFIDENTIAL 


CONFIDENTIAL 


PING  NU. 

stc. 

(S/N) 

utv 

1 

,4110 

16.46 

3.4 

d* 

i'*.':*!  1 

Ib.rtb 

-4.5 

J 

A 

b 

SA.Slb 

13.4b 

. b 

G 

3A.414 

ii.i<e 

-0.1 

7 

JA . 4] 4 

10.40 

3.0 

M 

3A.4d'G 

11.37 

. 1 

'4 

10 

11 

3<f  . ^744 

17.41 

-14.6 

Id- 

1 3 

34. 4^;  7 

10.41 

13.5 

lA 

i4.44  1 

10. Mi: 

?.b 

lb 

J4.44  7 

14.43 

-0.4 

IG 

1 7 

14.47 

-7.3 

IH 

34,44  7 

1 b.Od 

b.ri 

1'^ 

34. 4h  7 

?1  .^6 

-H.^ 

?l) 

34.4S4 

?1 .56 

?.b 

i>  llV"A  = n t 


Fig.  D-Bl 


: - 

\ 


CONFIDENTIAL 


t-  ■ r- 


r 


CONFIDENTIAL 


f\)»— ^ 

o^a-^acr#ufc*-o^£a^aLT 


o 

c> 


87 


o c o «v 


X X u) 


H- 


I 

n 


ro 

U) 

LD 

CD 

X 


ro 

O' 

OC  <J 

OD 

nC 


ro 

u 

j* 

■j* 

-g 

X 

X 


fo 

X 

-g 

a 

>c 


fv 

u 

■J^ 

X 

-g 

X 

o 


LT 

X 

-g 

X 

X 

o 

ro 

LJ 

LP 

cr 

-g 

X 

X 

o 

ro 

O’ 

X 

-g 

X 

X 

o 

ro 

U) 

JT 

X 

-g 

X 

X 

o 

V 

LO 

LT 

a 

(X 

X 

o 


r 

c 

£ 

> 

X 


c 

X 


n 

rr 

(X 

to 


X 

> 

2: 

C- 


> 

X 


CO 

UJ 

o 


IX 

x> 

X 

X 


cn 

j> 

2 

X 

1“ 

m 

i/) 


X 

> 

2 

C 


X 

2 


CP 


n 

CO 

« 

« 

« 

X 

X 

> 

CP 


c 

> 


-*  c 


o 

2 


CONFIDENTIAL 


"3 


CONFIDENTIAL 


O 88 


CONFIDENTIAL 


f\) 

cn 

u> 

X 

t/3 

r 

r 

X 

s 

>c 

"Z- 

X 

X 

t 

p 

c 

a 

X 

■>•>4 

•*» 

c 

c; 

** 

y 

> 

•— 

C' 

r 

« 

y. 

X 

J 

y 

J 

« 

“ 

X 

T 

I? 

o 

0 

►— 

— 

« 

X 

O' 

« 

Uo 

X 

r? 

X 

•s 

1 

J> 

'• 

y 

-s 

-r 

a 

X 

X 

j> 

c. 

~ 

X' 

rr 

y 

J* 

cr 

> 

X 

x: 

- 

>c 

o 

•-M 

r 

y 

X 

uJ 

IA' 

.P- 

UJ 

z 

o 

-«g 

X 

a 

•L 

y 

C 

t> 

*— ' 

3 

■»  v»- 

X 

T 

1 .•«.  -*« 

VjJ 

r" 

m 

L . 't 

LT 

y 

O' 

-4 

'« 

r - ■■  ; 

t > 

X 

’ t 

>c 

o 

'<» 

X 

** 

u; 

y 

c 

> 

lT 

3 

; 

X 

■c 

« 

-«w 

r 

X 

fT' 

r 

£ 

y 

l- 

X 

X 

n* 

u.' 

X 

■ 

f 

-T 

r 

- • 

-4 

X 

X 

c 

rr 

X 

X 

U) 

z 

>-  s^.'v: 

a- 

cr 

CD  uui 

-»w 

C-3  C1-. 

a 

lO 

o 

X 

u> 

r *v<  --5I 


iZ 


AO-A033  132 

UNCLASSIFIED 

20-^2 

AO 

A033I32 


UNITECH  INC  AUSTIN  TEX 

COMPUTER-AIDED-DETECTION  VALIDATION  STUDY. (U) 
NOV  69 

UNITECH-69-041-C 


F/6  9/2 


CONFIDENTIAL 


X 0 j * uiv^c 


V 

r 

c 

c •- 
»v 


c «- 


s w -- 
u 


a 

C.  J> 

rv  ^ Z. 

fV  T5 

uj  r 

S X 

O'  • 

\ 

U>  ‘X 

•c 


C-.3 


- 3 

i-..’  ? 

L.  = . ;•» 

c...  «.  4 


IV 

u;  </'. 

*> 

•J  3 

T V 

->j  r 

a ■'■ 

vC  lt. 


VT  IV  F' 

u z 

r 

O'  > 

^ z 

» <r 

sO  T 
o 


V — 

u z 


• ' '••"i 


mzC  ^ 


rT" 

CD  11  ^ 
CD  CX. 


CONFIDENTIAL 


710720  0375 


CONFIDENTIAL 


90 


•M. 

*'f  L • 

(S/M) 

Ol-  V 

I 

■''•1  1 1 

1 y . 3tt 

-b.-t’ 

. 1 ^ ^ 

1 / . H 6 

).b 

i 

i ‘ 1 

1 *♦ . /o 

-4.  J 

i H 

Z*  0 . '♦  i 

J 

) 

‘♦S 

.b 

/ 

< -'V  ,. 

(►.  -t 

t*-,  i 

1 / . < 0 

-# 

1 J.O/ 

l.ii 

■ 

t "•  . i ‘J 

) II  . MO 

-\^.l 

1 

T , ■)  r ' 

17.‘j0 

l«.o 

/ 

■*  * « ♦/"».* 

/O.^b 

1 .<< 

j 

1-“.  , M -•<' 

1 >".c'S 

-/.  1 

i 

' * , + ' > 

. -i‘+ 

M.o 

• ■> 

.'  •'  • ♦ 

W.Mb 

-H.") 

( 

V , -ir 

b.S 

V • -:^M 

l-^.ijd 

1.H 

- . - 

/I  . 

Fig.  D-B2 


1^'.  V • ' 


“ > ' 

V':- 


i j’ 


•v-U 


CONFIDENTIAL 


Fig.  D-C2 


CONFIDENTIAL 


91 


X 

•/ 

? 

I 

r 

r 

<* 

j 

cr 

3 

T 

X 

- 

-T 

J 

- 

“• 

« 

.X 

u 

y 

J 

m 

•■s. 

•“ 

y 

\ 

c 

« 

« 

rc 

« 

Wb' 

T 

3" 

T 

J- 

a 

J 

•• 

KT 

X 

I 

•— 

c 

C 

fV 

fT 

> 

U/ 

— 

y: 

X 

LT 

X 

X 

T* 

oc 

c 

>c 

3 

o 

M 

o 

*-• 

—1 

l\) 

U' 

o 

O' 

z 

LX- 

0" 

a* 

u 

a 

o 

> 

X 

r\j 

T 

n 

ji 

y> 

« 

X 

•)  . .'c 

>c 

..  i 3 

o 

X 

- ■« 

X 

t 

fv 

<r..;3 

Ul 

#• 

y) 

> 

3 

i ...'.i  .■  ■'» 
i •>  - J 

■d 

c 3 

r 

X 

rr 

c r-  t I 

c 

tA 

o 

*-•  • 

►-• 

X 

- L'J 

fV 

rr, 

‘ ' . . 1 

X 

m ' -i 

X 

' c,”-j 

a 

-s 

9 

» 

X 

o 

■ V..-J! 

i,i.; 

<C 

m 

o 

X 

...J 

r\j 

— .!  .—I 

U) 

z 

1 — 

X 

nD 

>- 

o 

CVmw 

rv 

CZ>  UU 

u> 

C4>  Om, 

CONFIDENTIAL 


B 


iXir- 


CONFIDENTIAL 


^c  a:  •>.»vr'»urv/*- 


^ J 3 


H- 

ua 


D 

I 

o 

to 


•D 

92 

f\j  *- 

C 

IV 

(/) 

u 

> 

X 

v/ 

iT 

r 

c 

a 

« 

o 

> 

z 

fV 

,T- 

u* 

c 

•f- 

a 

^ U>  O' 

X 

C“ 

rr 

J 

> 

fV 

r 

u* 

Uk- 

r» 

« 

y/ 

X 

J* 

y 

> 

• 

3 

D 

T 

»v 

c 

•— 

ru 

a 

« 

U/ 

X 

o 

X 

X 

X 

a 

« 

y 

T 

X 

X 

•— 

© 

Zi 

w 

c 

i\i 

rr 

u 

«N 

y 

—1 

X 

X 

•yy 

a 

•g 

c 

«c 

c 

o 

n 

y 

-g 

•>J 

w 

O 

(X 

X 

o* 

S 

-g 

u 

•X 

y 

o 

> 

3 

?v 

X 

u> 

r“ 

rr 

X 

y 

X* 

-g 

« 

««w- 

S--> 

o 

X 

■StIL.  ‘ 

1 ■- 

a 

f\) 

C'^  r “j 

Ul 

y 

i-j.j  > 

J* 

X 

> 

X 

X 

tr  :>  ,•  ' -7 

ti::  f •.> 

r 

t* 

a 

rr 

C.“-  t .,, 

c 

y 

era 

X 

C>  » ’ J 

O fV 

fT 

M -T 

u 

Jl 

z 

X 

cr.;  . .;] 

t . 

a 

> 

. L.4_f 

z 

X 

a 

■i  1 III  t 

T’ 

C-W 

I 

J 

.V 

»>>  . ^ 

UJ 

z 

■etc  "■■■  ^ 
^ 1— 

Oh 

-g 

tZT 

X 

o 

o 

O— 

f\J 

ca  loj 

U) 

^ r C3m 

CONFIDENTIAL 


xrrcsExai 


CONFIDENTIAL 


PING  MO 


(S/N) 


10 

11 

1? 

13 

14 

15 

16 
17 
IB 
I'* 

?0 

SIGmA 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


i 

i ‘ 


I jj 


3 

M 

o 

mm 

¥»* 

r\) 

t/i 

u> 

CD 

(j) 

m 

r 

o 

■X 

m 

o 

> 

X 

w 

#- 

Q 

J1 

D 

O' 

M 

X 

<f 

9 

“H 

yi 

M. 

•H 

X 

u 

w 

,V 

« 

;/5 

D 

/> 

£ 

> 

« 

*>1 

T 

T* 

'C 

« 

96 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 

98 


M 1 ' 

( S/  m) 

» i>“  V 

1 

IS,  - 

I '1 . 

3 

i '4  1 

vn.vi 

- 1 7 . y 

u 

■i  S , / i t 

1 / 

17.4 

■ 

-i**  , i 

1 n.^i; 

tm 

1 fi , ii 

- 1 . '• 

1 

1 . ■ 7 

1 ' 

1 * 

'It 

1 1 . 1 <i 

- ) 1 . • 

■*  ■ ; , .'  -S  1 

1 1 « 1 M 

IJ.ii 

\ ^ 
1 

f 

! 4 

>'1 

1 J.  • 

1 - 

^ t*  , t* 

^ ( . ''4 

''f 

1 • 

, *(:  / 

1 . 7'^ 

4.4' 

1 / 
) ^ 

< <4  , lx 

1 ' . - 7 

1 < 

i r*. 

- 4 . 4l 

Pi 

.■>  t 

■ 

h ^ 1 / , 1 k-4  14 

I <4  , J4 

-■■•  . 7 

Fig.  D-B4 


CONFIDENTIAL 


CONFIDENTIAL 


LU 
C_>  f-c. 


CONFIDENTIAL 


CONFIDENTIAL 


100 


f\i  ^ 


^ 25 

s 


j'  r 


X * 
-C 


o r 

2- 

V *■ 


CONFIDENTIAL 

102 


PIN«  ^|0. 

•^tc. 

( S/N) 

DEV 

1 

1 H 

1 3.  n 

i 

-T’  i 

• j 

? 

If'') 

1H.II7 

S.b 

— i 

20.«0 

28.-4 

4 

11.11 

-24.0 

b 

<<♦.  1 

If). 21 

-14.7 

h 

ISi) 

14.71 

1 

1 0.  IV 

-1H.2 

H 

^3.  -»'<-« 

14.  JU 

2.  1 

9 

:n.w?i 

».4l- 

I'rO- 

10 

1) 

3 1.771 

16.78 

IV. ^ 

1? 

3J.f)W  < 

1 3.V8 

31.7 

1 1 

M . ^ H 

10.48 

2'1.  H 

I't 

12.  73 

-o.b 

It) 

■n.-l.l  -■ 

1 1 .VO 

- 1 4 . •-< 

If) 

17 

11.^01 

13.41 

-1  1 .*1 

IH 

It* 

Vl.,'>')» 

12.7b 

-l.f 

20 

SJ  f-i'-tA 

= IM.;). 

Fig.  D-B5 


i 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


fliai  u*'^' 


'i 


CONFIDENTIAL 


CONFIDENTIAL 


■ --^ 
C*  , i . ■•% 

c.:.  iij 

...  -j 

C *"  t ■ ,:3 


tf'  •~-3 


CL3  ^ 


CONFIDENTIAL 


&>t2l0f>8i.9*i^t('  l UbWi.9^*^C2  I '<S#t2|06H/  VS^KiilObf'/  '^S^f  / I OM</  2 I 0bH/.‘*S*.r?lOhW/  u'^vv V I <th*<  / ^ 1 <)bW/  Ssiri  <^T 

NIH  -19NVU  t.  3ncll«.tfb  tOl  *S-lldlNVS  />.  bl  I'V'IS  ^‘>M*M  “’T  ‘^  • ■((  v<(|  moTT**  SllfllU 


CONFIDENTIAL 


1 I 


>V»  w w ^ 

a<ae-^9Ji«'U>i»f-'3£3B^  o>.  jt  » u<  i\i 

, _ ! ' I 


3 

C" 

t/» 

X 

m 

n 

X 


109 


c ^ 


3 


^ V c. 

£ £ 


IV 


J*  9 W 
► ^ 
J1 
J* 

X 

A 


M 

O CJ 
V #* 


V 


H* 

^a 


> 

«nI 


V 


O V 


o o — 


V 


9 

fV  V 


^9  O 

V 
9 


9 9 9 

9 

O 


J\ 

> 


o 

X 


X 

n 

i/» 


X 


M m 

i 2 

fr 

35  S 

«a 


9^  9 


fV 

u> 

>- 

JT 

X 

>o 

o 


Jt 

> 

X 


> 

-4 
X 

9 

M 
U 

Jl 
9^ 

-4 
X 
X 
o 
-*  X 
V mm 
4J  z 

LH 

9^ 


ro 

u) 

X 

X 


cn 


9 

vH 


Z 

9 


V 

m 

X 


z 

o 


o 

' 'X 


« 

9 

• 

X 

X 

> 

i/i 

m 

I o 

‘ z 

m 


X 

fT1 

o 

c 

n 


f!/0  C.-? 

cr.3 

£.  J C") 
u.-i 
c,':?  L..  - 
C-*i 

C-J  w J 

I »'•  «■ 

L- '2 

>— ! C:'.-» 

I,Ji~.1  . .«J 
-,..1 
r-  4 


.i 


CONFIDENTIAL 


<o 

9 


M 

U» 


tuz 

UUI 
CJ3  CX. 


<S/M) 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


i / c i 


V J 

^ y 


— r n 

V -« 


«■ 


r--  . .. 
r.. 

c ..  I 

\ 

C"*r  ' J 

K ] 

l-i-'i  ..J 

— I 

~ • K . 4 

SifS 

-«c  I — 

C:UZ 

c»  Uij 

CO  ct. 


CONFIDENTIAL 


CONFIDENTIAL 


r-'  ^ 


sr  r . 

(S/'i) 

l"  V 

1 

•1  •> , '.  1 ■ 

^'y.  •>h 

. / 

<■ 

•<-),  'f  ^ 

? 1 . <»  6 

/ 

3 

1 f' , 1 

) 7.-^,? 

y,y 

S 

. 1 1 1 

yi 

iH.'.i 

• 

/ 

•iS.  - ^ ^ 

1 U 1 1 

! • ' 

n 

■(  T , ‘i  ^ ^ 

1 

• '■» 

i 

X 

1 (' . -.<- 

1 " 

1 1 

■<  '•'.  "1 1 1 

1 '*  • 1 

• • I 

1 

1 H 

i M 

1 '< 

'iry 

1 ^ . 1 ^ 

1.1 

I '1 

1 / 

‘ 1 < 

1 ^ . Ui 

-M.** 

1 A 

. W 1 

1 ^ 

■»  *> , s / 

1 ‘1 . i ■* 

'i  . ■) 

J In-  ,i*  ” 

> s 

1 X.  (-.s 

Fig.  D-B8 


Fig.  D-D8 


CONFIDENTIAL 


< 


CONFIDENTIAL 


* ... 


I Ilf,--;  <-".  vl  »-  I "(  t-j  I'-.’/iv  1 1'f  >•>/  r Hie.w/.'J'- vvv  M h-<  / *'K'  I '•f'f-  / f!  t 


CONFIDENTIAL 


H- 


0 

1 

> 

VD 


X X ►- 


rc 


\ 


fv 


J J"  — - 


a 


^ ♦ — X X 


c a ^ >4: 


X 

>c  *C  X J 
t 

c c 


CONFIDENTIAL 


s 

t 

I 


117 


r. 

n 

(/ 

y 


X 

t> 

2 

(7 

rr 


> 

r 


wJ 

U 

o 


y> 

z 

X 


y> 


o 

■j‘ 

c/5 

Z 

X 


(X 

X 

r* 

X 

X 

> 

X 

C5 


X 

z 


y 


X 


o 

rv 


« 

X 

T 

y5 


> 

X 

X 

C' 

o 


r r. 

L 

•'-"  -•  -■'  VJ 

tT--.  \t 


fi 

C 

f:.. 

c 


r 


»V-Vf 

c_-:>  cx* 


CONFIDENTIAL 

118 


(S/  <1) 

Uf:  V 

1 

U.  , 1 

1 ti  • n J 

1 

• 

a 

t' 

1'  . * 1 1 

yi 

-^,1 

•i 

in.**!  p 

14. VJ 

is.n 

n.l 

^ ^ ^ 

?1 . 1)  3 

-0.3 

i 

IS. *17 

-S.  4 

t 

in.  7 1 1 

11.13 

1 ^ 

•n  . ^ 

1 n.nb 

■1 

in.  -p  1 ,1 

13. OH 

-in.n 

1 1. 

*n . 

li*.*! 

i i 

.in . -i><-) 

lS.o<* 

-O.b 

i'' 

->n.n! 

lb.  i 1 

-?.b 

1 .1 

* f . . . 1 

l/.b 

1 - 

<1..  'pn  , 

17.1b 

^>.4 

. •• 

1 V.-Pb 

1 1.1* 

1 ' ' 

in.  ? 1 1 

J ! 

In,  1 r'  1 

1 / . p>  / 

?.x) 

1 “ 

IX- , f ’■  1 

1 n.  'lO 

-4.0 

1 " 

* * , I 

1 b . '1  j 

-11.7 

1 

1 /:• 

/ 1 . -in 

4.  7 

CONFIDENTIAL 


\)  0mt  ^ ^ ^ ^ ^ ^ ^ ^ M 


^ X 

a 

j'  r 


V 

V.  ~:  -^y 

' 3> 

^ 'rN' 


CONFIDENTIAL 


HiNot  SIA»  I IS  34730  samples,  lof,  SAMPLtS  PtK  RANGE  HiN 


1 


7 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


(b/N) 


r.o 


Hb.  ->0  ! 

dt>. 

Hb,  33«; 

•t:?.  c'b-* 


CONFIDENTIAL 


CONFIDENTIAL 

j ^ 123 


CONFIDENTIAL 


Pig.  D-All 


CONFIDENTIAL 


3 

1 0^  ^0  ^ m* 

<C3c-~7a*#'U<i\j>'C<cX'g9vr>wr\>>' 

c. 

-4 

ro 

(/) 

u 

X 

rr 

O 

»M« 

O' 

n 

IV  fc 

o 

33 

X 

<c 

Oj 

9 

> 

r 

ro 

't; 

o 

LO 

U)  o 

c o ^ w 

#> 

C 

C Iji  >i.  « « £ 

ir 

a 

♦■  -c  ^ 

3^ 

2 

z 

o 

>C 

c 

rr 

U) 

•-  f\J 

o 

>«. 

■H 

o 

o c 

ro 

r> 

UI 

fT 

« 

X 

Ut 

t/l 

•M 

o 

^ vu 

• 

Ji 

w 

“ 

rt 

mP 

<c 

« 

o 

c 

9 

ip 

c 

N— 

H-* 

« 

►-* 

ro 

« 

00 

X 

Q 

I 

‘X 

X 

> 

O *-* 

O' 

« 

cn 

o 

m 

X 

X 

c 

> 

c 

e 

o 

z 

ro 

*«• 

C: 

c 

o 

ro 

rr 

> 

»>« 

Ob) 

o *- 

s 

cn 

> 

o 

o 

u> 

3^ 

> 

r 

'O 

X 

o o 

X 

•H 

o 

£ 

o 

n 

3) 

^) 

ro 

o 

OJ 

UI 

o 

UI 

z 

rsj 

X 

> 

1— » 

*>0 

c o 

X 

£ 

X 

o 

o 

> 

z 

IT 

o 

fO 

X 

u 

n 

?Ti 

'N) 

CO 

O' 

« 

O 

o 

X 

ro 

>o 

pp« 

o 

O f>) 

9 

ro 

o o 

U) 

(/» 

UI  ^ 

#• 

X 

> 

X 

-0 

r 

o 

X 

m 

o 

U) 

ro 

o 

X 

ro 

u> 

X 

o 

.11 

s 

— o 

-o 

z 

o 

X 

6> 

o 

rt» 

1—  o 

o 

X 

o 

ro 

UI 

z 

u* 

o> 

o •- 

r\) 

s 

« 

CJ 

o 

e 

w 

ro 

ro 

0 9 M 

ro  ro 

.p* 

125 


r-o  k.  ' 

I j ; 
r. 

c::.’. 

c." 

r. 

<r‘- 

j — 

tr-  ' ■ 


jUjl  j 


CONFIDENTIAL 


w 


CONFIDENTIAL 


126 


PING  NO. 

stc. 

(S/N) 

UEO 

1 

3b. 40b 

14.30 

-4.4 

? 

3b. 41  1 

11.89 

.3 

3 

3b. 438 

11.0b 

-6.0 

4 

3b. 430 

?3.4b 

18. b 

3b. 467 

po.ya 

-2.8 

6 

3b. 480 

19. bl 

-0.3 

/ 

3b.49y 

19.68 

-3.7 

K 

3b. -)1  3 

1?.<'6 

-2.2 

y 

10 

3b. b4  7 

16.12 

-b.3 

1 1 

3b. bb/ 

??.y^ 

. 1 

3b.bh7 

17.9b 

b.b 

13 

3b.b86 

P1.90 

i.y 

14 

3b. 60b 

14.94 

-2.8 

lb 

3b.biy 

9.69 

-0.4 

Ih 
1 / 
IH 

ly 

35.b7H» 

9.69 

.6 

?o 

Sl(s-1A 

= S.b'JH 

Fig.  D-Bll 


CONFIDENTIAL 


CONFIDENTIAL 


CONFIDENTIAL 


